Using Machine-Learning to Optimize phase contrast in a Low-Cost Cellphone Microscope

Cellphones equipped with high-quality cameras and powerful CPUs as well as GPUs are widespread. This opens new prospects to use such existing computational and imaging resources to perform medical diagnosis in developing countries at a very low cost. Many relevant samples, like biological cells or waterborn parasites, are almost fully transparent. As they do not exhibit absorption, but alter the light's phase only, they are almost invisible in brightfield microscopy. Expensive equipment and procedures for microscopic contrasting or sample staining often are not available. By applying machine-learning techniques, such as a convolutional neural network (CNN), it is possible to learn a relationship between samples to be examined and its optimal light source shapes, in order to increase e.g. phase contrast, from a given dataset to enable real-time applications. For the experimental setup, we developed a 3D-printed smartphone microscope for less than 100 \$ using off-the-shelf components only such as a low-cost video projector. The fully automated system assures true Koehler illumination with an LCD as the condenser aperture and a reversed smartphone lens as the microscope objective. We show that the effect of a varied light source shape, using the pre-trained CNN, does not only improve the phase contrast, but also the impression of an improvement in optical resolution without adding any special optics, as demonstrated by measurements

Using Machine-Learning to Optimize phase contrast in a Low-Cost Cellphone Microscope

Cellphones equipped with high-quality cameras and powerful CPUs as well as GPUs are widespread. This opens new prospects to use such existing computational and imaging resources to perform medical diagnosis in developing countries at a very low cost. Many relevant samples, like biological cells or waterborn parasites, are almost fully transparent. As they do not exhibit absorption, but alter the light's phase only, they are almost invisible in brightfield microscopy. Expensive equipment and procedures for microscopic contrasting or sample staining often are not available. By applying machine-learning techniques, such as a convolutional neural network (CNN), it is possible to learn a relationship between samples to be examined and its optimal light source shapes, in order to increase e.g. phase contrast, from a given dataset to enable real-time applications. For the experimental setup, we developed a 3D-printed smartphone microscope for less than 100 \$ using off-the-shelf components only such as a low-cost video projector. The fully automated system assures true Koehler illumination with an LCD as the condenser aperture and a reversed smartphone lens as the microscope objective. We show that the effect of a varied light source shape, using the pre-trained CNN, does not only improve the phase contrast, but also the impression of an improvement in optical resolution without adding any special optics, as demonstrated by measurements

Defining a Superlens Operating Regime for Imaging Fluorescent Molecules

It has been shown that thin metal-based films can at certain frequencies act as planar near-field lenses for certain polarization components. A desirable property of such “lenses” is that they can also enhance and focus some large transverse spatial frequency components which contain sub-diffraction limit details. Over the last decade there has been much work in optimizing designs to reduce effects (such as material losses and surface roughness) that are detrimental to image reconstruction. One design that can reduce some of these undesirable effects, and which has received a fair amount of attention recently, is the stacked metal-dielectric superlens. Here we theoretically explore the imaging ability of such a design for the specific purpose of imaging a fluorescent dye (the common bio-marker GFP) in the vicinity of the superlens surface. Our calculations take into consideration the interaction (damping) of an oscillating electric dipole with the metallic layers in the superlens. We also assume a Gaussian frequency distribution spectrum for the dipole. We treat the metallic-alloy and dielectric-alloy layers separately using an appropriate effective medium theory. The transmission properties are evaluated via Transfer matrix (-matrix) calculations that were performed in the MatLab and MathCad environments. Our study shows that it is in principle possible to image fluorescent molecules using a simple bilayer planar superlens. We find that optimal parameters for such a superlens occur when the peak dipole emission-frequency is slightly offset from the Surface Plasmon resonance frequency of the metal-dielectric interfaces. The best resolution is obtained when the fluorescent molecules are not too close ( nm) or too far ( nm) from the superlens surface. The realization and application of a superlens with the specified design is possible using current nanofabrication techniques. When combined with e.g. a sub-wavelength grating structure (such as in the far-field superlens design previously proposed [1]) or a fast near-field scanning probe, it could provide a means for fast fluorescent imaging with sub-diffraction limit resolution

One-shot phase-recovery using a cellphone RGB camera on a Jamin-Lebedeff microscope

Jamin-Lebedeff (JL) polarization interference microscopy is a classical method for determining the change in the optical path of transparent tissues. Whilst a differential interference contrast (DIC) microscopy interferes an image with itself shifted by half a point spread function, the shear between the object and reference image in a JL-microscope is about half the field of view. The optical path difference (OPD) between the sample and reference region (assumed to be empty) is encoded into a color by white-light interference. From a color-table, the Michel-Levy chart, the OPD can be deduced. In cytology JL-imaging can be used as a way to determine the OPD which closely corresponds to the dry mass per area of cells in a single image. Like in other interference microscopy methods (e.g. holography), we present a phase retrieval method relying on single-shot measurements only, thus allowing real-time quantitative phase measurements. This is achieved by adding several customized 3D-printed parts (e.g. rotational polarization-filter holders) and a modern cellphone with an RGB-camera to the Jamin-Lebedeff setup, thus bringing an old microscope back to life. The algorithm is calibrated using a reference image of a known phase object (e.g. optical fiber). A gradient-descent based inverse problem generates an inverse look-up-table (LUT) which is used to convert the measured RGB signal of a phase-sample into an OPD. To account for possible ambiguities in the phase-map or phase-unwrapping artifacts we introduce a total-variation based regularization. We present results from fixed and living biological samples as well as reference samples for comparison

Better than a lens -- Increasing the signal-to-noise ratio through pupil splitting

Lenses are designed to fulfill Fermats principle such that all light interferes constructively in its focus, guaranteeing its maximum concentration. It can be shown that imaging via an unmodified full pupil yields the maximum transfer strength for all spatial frequencies transferable by the system. Seemingly also the signal-to-noise ratio (SNR) is optimal. The achievable SNR at a given photon budget is critical especially if that budget is strictly limited as in the case of fluorescence microscopy. In this work we propose a general method which achieves a better SNR for high spatial frequency information of an optical imaging system, without the need to capture more photons. This is achieved by splitting the pupil of an incoherent imaging system such that two (or more) sub-images are simultaneously acquired and computationally recombined. We compare the theoretical performance of split pupil imaging to the non-split scenario and implement the splitting using a tilted elliptical mirror placed at the back-focal-plane (BFP) of a fluorescence widefield microscope

simpleISM—A straight forward guide to upgrade from confocal to ISM

Resolution in a confocal laser scanning microscopes (CLSM) can be improved if the pinhole is closed. But closing the pinhole will deteriorate the signal to noise ratio (SNR). A simple technique to improve the SNR while keeping the resolution same by upgrading the system to an image scanning microscope. In this paper, we explain in detail, based on an Olympus Fluoview 300 system, how a scanning microscope can be upgraded into an image scanning microscope (ISM) using a simple camera-based detector and an Arduino Due providing a galvo driving and camera synchronization signals. We could confirm a resolution improvement as well as superconcentration and made the interesting observation of a reduced influence of laser fluctuations

Thermal illumination limits in 3D Raman microscopy: A comparison of different sample illumination strategies to obtain maximum imaging speed

Confocal Raman microscopy is a powerful tool for material science and biomedical research. However, the low Raman scattering cross-section limits the working speed, which reduces the applicability for large and sensitive samples. Here, we discuss the fundamental physical limits of Raman spectroscopy with respect to signal-to-noise, sample load and how to achieve maximal imaging speed. For this, we develop a simple model to describe arbitrary far field light microscopes and their thermal influence on the sample. This model is used to compare the practical applicability of point- and line-confocal microscopes as well as wide-field-, light sheet- and light line illumination, for the measurement of 3D biological samples. The parallelization degree of the illumination can positively affect the imaging speed as long as it is not limited by thermal sample heating. In case of heat build-up inside the sample, the advantages of parallelization can be lost due to the required attenuation of excitation and the working speed can drop below that of a sequential method. We show that for point like illumination, the exposure time is thermally not as critical for the sample as the irradiance, while for volume like illumination, the exposure time and irradiance result in the same thermal effect. The results of our theoretical study are experimentally confirmed and suggest new concepts of Raman microscopy, thus extending its applicability. The developed model can be applied to Raman imaging as well as to other modes (e.g. two- or three- photon imaging, STED, PALM/STORM, MINFLUX) where thermal effects impose a practical limit due to the high irradiance required

cellSTORM - Cost-effective Super-Resolution on a Cellphone using dSTORM

Expensive scientific camera hardware is amongst the main cost factors in modern, high-performance microscopes. Recent technological advantages have, however, yielded consumer-grade camera devices that can provide surprisingly good performance. The camera sensors of smartphones in particular have benefited of this development. Combined with computing power and due to their ubiquity, smartphones provide a fantastic opportunity for "imaging on a budget". Here we show that a consumer cellphone is capable even of optical super-resolution imaging by (direct) Stochastic Optical Reconstruction Microscopy (dSTORM), achieving optical resolution better than 80 nm. In addition to the use of standard reconstruction algorithms, we investigated an approach by a trained image-to-image generative adversarial network (GAN). This not only serves as a versatile technique to reconstruct video sequences under conditions where traditional algorithms provide sub-optimal localization performance, but also allows processing directly on the smartphone. We believe that "cellSTORM" paves the way for affordable super-resolution microscopy suitable for research and education, expanding access to cutting edge research to a large community

Optical Photon Reassignment Microscopy (OPRA)

To enhance the resolution of a confocal laser scanning microscope the additional information of a pinhole plane image taken at every excitation scan position can be used [C. J. R. Sheppard, Super-resolution in confocal imaging, Optik 80, 5354 (1988)]. This photon reassignment principle is based on the fact that the most probable position of an emitter is at half way between the nominal focus of the excitation laser and the position corresponding to the (off centre) detection position. Therefore, by reassigning the detected photons to this place, an image with enhanced detection efficiency and resolution is obtained. Here we present optical photon reassignment microscopy (OPRA) which realises this concept in an all-optical way obviating the need for image-processing. With the help of an additional intermediate optical beam expansion between descanning and a further rescanning of the detected light, an image with the advantages of photon reassignment can be acquired. Due to its simplicity and flexibility this method has the potential to enhance the performance of nearly every laser scanning microscope and is therefore expected to play an important role in future systems.Comment: 8 pages, 3 figure

Motion artefact detection in structured illumination microscopy for live cell imaging

The reconstruction process of structured illumination microscopy (SIM) creates substantial artefacts if the specimen has moved during the acquisition. This reduces the applicability of SIM for live cell imaging, because these artefacts cannot always be recognized as such in the final image. A movement is not necessarily visible in the raw data, due to the varying excitation patterns and the photon noise. We present a method to detect motion by extracting and comparing two independent 3D wide-field images out of the standard SIM raw data without needing additional images. Their difference reveals moving objects overlaid with noise, which are distinguished by a probability theory-based analysis. Our algorithm tags motion-artefacts in the final high-resolution image for the first time, preventing the end-user from misinterpreting the data. We show and explain different types of artefacts and demonstrate our algorithm on a living cell